Valve controlled, node-level vapor condensation for two-phase heat sink(s)

ABSTRACT

A cooling apparatus and method are provided for cooling one or more electronic components of an electronic subsystem of an electronics rack. The cooling apparatus includes a heat sink, which is configured to couple to an electronic component, and which includes a coolant-carrying channel for coolant to flow therethrough. The coolant provides two-phase cooling to the electronic component, and is discharged from the heat sink as coolant exhaust which comprises coolant vapor to be condensed. The cooling apparatus further includes a node-level condensation module, associated with the electronic subsystem, and coupled in fluid communication with the heat sink to receive the coolant exhaust from the heat sink. The condensation module is liquid-cooled, and facilitates condensing of the coolant vapor in the coolant exhaust. A controller automatically controls the liquid-cooling of the heat sink and/or the liquid-cooling of the node-level condensation module.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. Ser. No. 13/189,598, entitled“Valve Controlled, Node-Level Vapor Condensation for Two-Phase HeatSink(s),” filed Jul. 25, 2011, and which is hereby incorporated hereinby reference in its entirety.

BACKGROUND

The power dissipation of integrated circuit chips, and the modulescontaining the chips, continues to increase in order to achieveincreases in processor performance. This trend poses a cooling challengeat both the module and system level. Increased airflow rates are neededto effectively cool high power modules and to limit the temperature ofthe air that is exhausted into the computer center.

In many large server applications, processors along with theirassociated electronics (e.g., memory, disk drives, power supplies, etc.)are packaged in removable node configurations stacked within a rack orframe. In other cases, the electronics may be in fixed locations withinthe rack or frame. Typically, the components are cooled by air moving inparallel airflow paths, usually front-to-back, impelled by one or moreair moving devices (e.g., fans or blowers). In some cases it may bepossible to handle increased power dissipation within a single node byproviding greater airflow, through the use of a more powerful air movingdevice or by increasing the rotational speed (i.e., RPMs) of an existingair moving device. However, this approach is becoming problematic at therack level in the context of a computer installation (i.e., datacenter).

The sensible heat load carried by the air exiting the rack is stressingthe ability of the room air-conditioning to effectively handle the load.This is especially true for large installations with “server farms” orlarge banks of computer racks close together. In such installations,liquid cooling (e.g., water cooling) is an attractive technology tomanage the higher heat fluxes. The liquid absorbs the heat dissipated bythe components/modules in an efficient manner. Typically, the heat isultimately transferred from the liquid to an outside environment,whether air or other liquid coolant.

BRIEF SUMMARY

In one aspect, a method of facilitating extraction of heat from aheat-generating electronic component is provided. The method includes:providing at least one heat sink configured to cool at least oneelectronic component of the electronic subsystem of an electronics rack,the at least one heat sink comprising at least one coolant-carryingchannel configured for coolant to flow therethrough, the coolantproviding two-phase cooling to the at least one electronic component andbeing discharged from the at least one heat sink as coolant exhaust withcoolant vapor; providing a node-level condensation module in associationwith the electronic subsystem, and coupling in fluid communication theat least one heat sink and the node-level condensation module, thenode-level condensation module receiving the coolant exhaust from the atleast one heat sink, the node-level condensation module beingliquid-cooled and facilitating condensing of the coolant vapor in thecoolant exhaust; and automatically controlling at least one of theliquid-cooling of the at least one heat sink or the liquid-cooling ofthe node-level condensation module.

Additional features and advantages are realized through the techniquesof the present invention. Other embodiments and aspects of the inventionare described in detail herein and are considered part of a the claimedinvention.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

One or more aspects of the present invention are particularly pointedout and distinctly claimed as examples in the claims at the conclusionof the specification. The foregoing and other objects, features, andadvantages of the invention are apparent from the following detaileddescription taken in conjunction with the accompanying drawings inwhich:

FIG. 1 is an elevational view of one embodiment of a liquid cooledelectronics rack comprising one or more heat-generating electroniccomponents, and employing one or more heat sinks, in accordance with oneor more aspects of the present invention;

FIG. 2 is a schematic of one embodiment of an electronic subsystem ornode of an electronics rack, wherein an electronic component andassociated heat sink are cooled by system coolant provided by one ormore modular cooling units disposed within the electronics rack, inaccordance with one or more aspects of the present invention;

FIG. 3 is a schematic of one embodiment of a modular cooling unit for acooled electronics rack such as depicted in FIGS. 1 & 2, in accordancewith one or more aspects of the present invention;

FIG. 4 is a plan view of one embodiment of an electronic subsystemlayout illustrating multiple heat sinks cooling multiple electroniccomponents of the electronic subsystem, in accordance with one or moreaspects of the present invention;

FIG. 5A is a cross-sectional elevational view of one embodiment of acooled electronic structure comprising a heat-generating electroniccomponent and a heat sink with a vapor-permeable membrane, and takenalong lines 5A-5A in FIGS. 5B & 5D, in accordance with one or moreaspects of the present invention;

FIG. 5B is a cross-sectional plan view of the cooled electronicstructure of FIG. 5A, taken along line 5B-5B thereof, in accordance withone or more aspects of the present invention;

FIG. 5C is a cross-sectional plan view of the cooled electronicstructure of FIG. 5A, taken along line 5C-5C thereof, in accordance withone or more aspects of the present invention;

FIG. 5D is a cross-sectional plan view of the cooled electronicstructure of FIG. 5A, taken along line 5D-5D thereof, in accordance withone or more aspects of the present invention;

FIG. 6 is a schematic one embodiment of a cooled electronics apparatuscomprising an electronic subsystem or node with multiple heat sinks andillustrating controlled node-level vapor condensing, in accordance withone or more aspects of the present invention;

FIG. 7 depicts one embodiment of a control process for adjusting coolantflow between one or more heat sink structures and the node-levelcondensation module within a cooled electronics apparatus such asdepicted in FIGS. 5A-6, in accordance with one or more aspects of thepresent invention;

FIG. 8 is a schematic of one embodiment of a direct-cooled, node-levelcondensation module employed, by way of example, in the cooledelectronics apparatus of FIG. 6, in accordance with one or more aspectsof the present invention;

FIG. 9 is a schematic of an alternate embodiment of a cooled electronicsapparatus comprising an electronic subsystem or node with multiple heatsinks, and illustrating an alternate embodiment of a node-levelcondensation module, in accordance with one or more aspects of thepresent invention;

FIG. 10 depicts one embodiment of a control process for adjusting flowof facility coolant to the node-level condensation modules of a cooledelectronics apparatus such as depicted in FIG. 9, in accordance with oneor more aspects of the present invention;

FIG. 11 is a schematic of one embodiment of an indirect-cooled,node-level condensation module for the cooled electronics apparatus ofFIG. 9, in accordance with one or more aspects of the present invention;

FIG. 12 is a schematic of another embodiment of a cooled electronicsapparatus comprising an electronic subsystem or node with multiple heatsinks and indirect-cooled, node-level vapor-condensing, in accordancewith one or more aspects of the present invention;

FIG. 13 is a cross-sectional elevational view of another embodiment of acooled electronic structure which may be employed within a cooledelectronics apparatus, in accordance with one or more aspects of thepresent invention; and

FIG. 14 is a cross-sectional elevational view of further embodiment of acooled electronic structure which may be employed in a cooledelectronics apparatus, in accordance with one or more aspects of thepresent invention.

DETAILED DESCRIPTION

As used herein, the terms “electronics rack” and “rack unit” are usedinterchangeably, and unless otherwise specified include any housing,frame, rack, compartment, blade server system, etc., having one or moreheat generating components of a computer system or electronics system,and may be, for example, a stand alone computer processor having high,mid or low end processing capability. In one embodiment, an electronicsrack may comprise a portion of an electronic system, a single electronicsystem or multiple electronic systems, for example, in one or moresub-housings, blades, books, drawers, nodes, compartments, etc., havingone or more heat-generating electronic components disposed therein. Anelectronic system(s) within an electronics rack may be movable or fixedrelative to the electronics rack, with rack-mounted electronic drawersand blades of a blade center system being two examples of electronicsystems (or subsystems) of an electronics rack to be cooled.

“Electronic component” refers to any heat generating electroniccomponent of, for example, a computer system or other electronic systemrequiring cooling. By way of example, an electronic component maycomprise one or more integrated circuit dies, chips, modules and/orother heat-generating electronic devices to be cooled, such as one ormore processors, memory modules and/or memory support structures.Further, as used herein, the terms “heat sink” and “coolant cooled heatsink” refer to thermally conductive structures having one or morechannels (or passageways) form therein or passing therethrough, whichfacilitate the flow of coolant through the structure. One example, thecoolant carrying channels comprise microchannels having a characteristicdimension of 1.0 mm or less, for example, in the range of approximately0.1 mm to 0.5 mm.

As used herein, “liquid-to-liquid heat exchanger” may comprise, forexample, two or more coolant flow paths, formed of thermally conductivetubings (such as copper or other tubing) in thermal or mechanicalcontact with each other. Size, configuration and construction of theliquid-to-liquid heat exchanger can vary without departing from thescope of the invention disclosed herein. Further, “data center” refersto a computer installation containing one or more electronics racks tobe cooled. As a specific example, a data center may include one or morerows of rack-mounted computing units, such as server units.

By way of example, the first coolant and the second coolant discussedherein are a system (or working) coolant, and a facility (or housecoolant), respectively. In one specific example, the system coolant andthe facility coolant are water. However, the concepts disclosed hereinare readily adapted to use with other types of coolant on the facilityside and/or on the system side. For example, one or more of the coolantsmay comprise a brine, a dielectric liquid, a fluorocarbon liquid, aliquid metal, or other similar coolant, or refrigerant, while stillmaintaining the advantages and unique features of the present invention.

Reference is made below to the drawings (which are not drawn to scalefor ease of understanding), wherein the same reference numbers usedthroughout different figures designate the same or similar components.

FIG. 1 depicts one embodiment of a liquid-cooled electronics rack 100which employs a liquid-based cooling system. In one embodiment,liquid-cooled electronics rack 100 comprises a plurality of electronicsubsystems or nodes 110, which may comprise processor or server nodes,as well as a disk enclosure structure 111. In this example, a bulk powerassembly 120 is disposed at an upper portion of liquid-cooledelectronics rack 100, and two modular cooling units (MCUs) 130 aredisposed in a lower portion of the liquid-cooled electronics rack. Inthe embodiments described herein, the coolant is assumed to be water oran aqueous-based solution, again, by way of example only.

In addition to MCUs 130, the cooling system includes a system watersupply manifold 131, a system water return manifold 132, andmanifold-to-node fluid connect hoses 133 coupling system water supplymanifold 131 to electronics structures 110, 111 and node-to-manifoldfluid connect hoses 134 coupling the individual electronics subsystems110, 111 to system water return manifold 132. Each MCU 130 is in fluidcommunication with system water supply manifold 131 via a respectivesystem water supply hose 135, and each MCU 130 is in fluid communicationwith system water return manifold 132 via a respective system waterreturn hose 136.

As illustrated, heat load of the electronic structures is transferredfrom the system water to cooler facility water supplied by facilitywater supply line 140 and facility water return line 141 disposed, inthe illustrated embodiment, in the space between a raised floor 145 anda base floor 165.

FIG. 2 schematically illustrates operation of the cooling system of FIG.1, wherein a liquid-cooled heat sink 200 is shown coupled to anelectronic component 201 of an electronic subsystem 110 within theelectronics rack 100. Heat is removed from electronic component 201 viathe system coolant circulated via pump 220 through heat sink 200 withinthe system coolant loop defined by liquid-to-liquid heat exchanger 221of modular cooling unit 130, lines 222, 223 and heat sink 200. Thesystem coolant loop and modular cooling unit are designed to providecoolant of a controlled temperature and pressure, as well as controlledchemistry and cleanliness to the electronic component(s). Furthermore,the system coolant is physically separate from the less controlledfacility coolant in lines 140, 141, to which heat is ultimatelytransferred.

FIG. 3 depicts a more detailed embodiment of a modular cooling unit 130,in accordance with an aspect of the present invention. As shown in FIG.3, modular cooling unit 130 includes a facility coolant loop whereinbuilding chilled, facility coolant is supplied 310 and passes through acontrol valve 320 driven by a motor 325. Valve 320 determines an amountof facility coolant to be passed through liquid-to-liquid heat exchanger221, with a portion of the facility coolant possibly being returneddirectly via a bypass orifice 335. The modular cooling unit furtherincludes a system coolant loop with a reservoir tank 340 from whichsystem coolant is pumped, either by pump 350 or pump 351, into the heatexchanger 221 for conditioning and output thereof, as cooled systemcoolant to the associated rack unit to be cooled. The cooled systemcoolant is supplied to the system supply manifold and system returnmanifold of the liquid-cooled electronics rack via the system watersupply hose 135 and system water return hose 136.

FIG. 4 depicts one embodiment of an electronic subsystem 110 layoutwherein one or more air moving devices 411 provide forced air flow 415to cool multiple devices 412 within electronic subsystem 110. Cool airis taken in through a front 431 and exhausted out a back 433 of thedrawer. The multiple devices to be cooled include multiple processormodules to which coolant-cooled heat sinks 420 (of a cooling system) arecoupled, as well as multiple arrays of memory modules 430 (e.g., dualin-line memory modules (DIMMs)) and multiple rows of memory supportmodules 432 (e.g., DIMM control modules) to which air-cooled heat sinksare coupled. In the embodiment illustrated, memory modules 430 and thememory support modules 432 are partially arrayed near front 431 ofelectronic subsystem 110, and partially arrayed near back 433 ofelectronic subsystem 110. Also, in the embodiment of FIG. 4, memorymodules 430 and memory support modules 432 are cooled by air flow 415across the electronic subsystem.

The illustrated liquid-based cooling system further includes multiplecoolant-carrying tubes connected to and in fluid communication withcoolant-cooled heat sinks 420. The coolant-carrying tubes comprise setsof coolant-carrying tubes, with each set including (for example) acoolant supply tube 440, a bridge tube 441 and a coolant return tube442. In this example, each set of tubes provides liquid coolant to aseries-connected pair of heat sinks 420 (coupled to a pair of processormodules). Coolant flows into a first heat sink of each pair via thecoolant supply tube 440 and from the first heat sink to a second heatsink of the pair via bridge tube or line 441, which may or may not bethermally conductive. From the second heat sink of the pair, coolant isreturned through the respective coolant return tube 442. In an alternateimplementation, tubing is provided for separately passing coolant inparallel through the heat sinks of the electronic subsystem.

In one embodiment, the above-described cooling system can be employedwith single-phase liquid-cooling. However, such a system requires alarge liquid flowrate, and correspondingly large, high-power pumps, toavoid the liquid boiling and minimize sensible heating of the fluid asit absorbs the heat dissipated. The flowrate and pump power required maybe reduced by an order of magnitude by leveraging the large latent heatof vaporization, allowing the liquid to boil. Flow boiling provides highheat transfer coefficients, which can facilitate reducing thejunction-to-ambient module thermal resistance, and can couple the moduletemperature to the boiling point of the coolant (or working fluid),resulting in better temperature uniformity.

However, flow boiling in the confined flow geometries of small heat sinkchannels, and small impingement jets in the heat sink, result in adetrimental rise in pressure due to bubble nucleation, bubble growth andadvection of the vapor phase. The rise in pressure shifts saturationconditions, delaying the onset of boiling, and also results in thedevelopment of flow instabilities and flow maldistribution at the heatsink and node level, which can lead to premature liquid dryout. Theseissues have made flow boiling microstructures difficult to implement.

As used herein, a “microchannel”, “micro-jet” or “microstructure” refersto a structure having a characteristic dimension less than 1.0 mm, forexample, of approximately 0.5 mm or less. In one implementation, themicrochannel has a characteristic dimension of approximately 100microns, and the jet channel (or jet orifice) has a diameter less than100 microns. In the implementations described herein, the jet orificediameter is assumed to be less than the microchannel width, since thejet orifice injects coolant into the microchannel(s) of the heat sink.

Disclosed hereinbelow are various heat sink structures which combinelocal jet impingement of coolant (through jet nozzles (or jet orifices))with local vapor removal via a porous, vapor-permeable membrane, whichminimizes the various challenges encountered during flow boiling inmicrostructures. The microchannels provide a larger heat transfer areaand improved thermal performance as compared to larger, conventionalchannels, and by incorporating a vapor-permeable membrane within theheat sink structure, vapor generated within the microchannels can escapethe confined microchannel geometry directly into a separate vaportransport channel/plenum. This local removal of vapor provides severaladvantages, including: a reduced two-phase flow pressure drop and areduced required pumping power for circulating coolant through the heatsink structure(s); a lower and more uniform coolant saturationtemperature within the heat sink structure; an improved heat transfercoefficient and reduced heat sink thermal resistance due to phasechange; improved wetting and improved jet impingement; and a reducedpossibility of flow instabilities which might lead to premature dryoutwithin the heat sink.

The separated vapor can be reintroduced into the coolant exhaust fromthe cooling microchannels within the heat sink structure itself, or at anode level within an electronics rack comprising the heat sinkstructure. As described herein, the coolant vapor is advantageouslycondensed at the electronic subsystem (or node level) through eitherdirect contact condensation with a diverted fraction of the sub-cooledcoolant employed within the heat sink(s), or indirect condensation usinga facility chilled coolant, such as facility chilled water, that isplumbed to the node-level condensation module (i.e., condenser). Thevapor is condensed back to liquid, and rejoins the liquid coolantreturning to the modular cooling unit, where the liquid can be cooledand pumped back to the nodes of the electronics rack. In one embodiment,the coolant flowing through the heat sink structures comprises water,and the membrane is a porous, hydrophobic membrane. Further, in oneembodiment, the membrane may be modified to have a spatially-varyingporosity and stiffness, which allows for both the injection of fluid,through jet orifices provided in rigid portions of the membrane, andlocal removal of vapor generated within the microchannels.Alternatively, a plate mask could be associated with the vapor-permeableregion of the membrane to define a multilayer structure, which comprisesone or more coolant injection regions and one or more vapor removalregions from the microchannels. Note that in the embodiments describedherein, the membrane, or the membrane and plate mask structure, overlieand form part of the coolant-carrying channels so as to be exposed tovapor within the coolant-carrying channels of the heat sink. Forexample, in one embodiment, the membrane forms a top portion of each ofthe coolant-carrying channels of the heat sink.

FIGS. 5A-5D depict one embodiment of a cooled electronic structure,generally denoted 500, in accordance with one or more aspects of thepresent invention. Cooled electronic structure 500 includes, in thisembodiment, an electronic component 510, such as an electronic module,mounted to a printed circuit board 501 with an associated back plate 502(for example, a metal back plate). A heat sink 503 is mechanicallycoupled via securing mechanisms 505 to back plate 502 of printed circuitboard 501, which provide a compressive load forcing heat sink 503 ingood thermal contact with electronic component 510. Electronic component510 includes, in this embodiment, an integrated circuit chip 511connected to a chip carrier or substrate 513 via, for example, a firstplurality of solder ball connections 512. Similarly, substrate 513 iselectrically connected to printed circuit board 501 via, for example, asecond plurality of solder ball connections 514. A thermally conductivecap 516 is interfaced to integrated circuit chip 511 via a first thermalinterface material 515, such as a silicone-based paste or grease, pad,epoxy or solder. A second thermal interface material 517 facilitatesthermal interfacing of cap 516 to heat sink 503.

In this embodiment, heat sink 503 comprises a multilayer heat sink witha heat sink base 521, a membrane structure 520 and a heat sink cap 530,which are respectively depicted in cross-sectional plan view in FIGS.5B-5D. Referring collectively to FIGS. 5A-5D, heat sink base 521comprises one or more coolant-carrying channels 522, each of which maycomprise a microchannel structure, such as described above. Note thatfive coolant-carrying microchannels are depicted in FIG. 5B, by way ofexample only. More or less coolant-carrying channels may be definedwithin the heat sink base, as desired. Heat from the electroniccomponent is rejected to coolant within the coolant-carrying channels inthe heat sink base. Two-phase cooling of the heat-generating electroniccomponent is achieved by at least partial vaporization of the coolant(i.e., working fluid) within the one or more coolant-carrying channelsof the heat sink.

As illustrated in FIGS. 5A & 5C, various regions of the coolant-carryingchannels are capped by at least one vapor-permeable region 523 ofmembrane structure 520. As illustrated in FIGS. 5A & 5D, disposed overthese regions are vapor transport channels 531 formed in heat sink cap530. Thus, localized venting of vapor directly from the coolant-carryingchannels, across the vapor-permeable membrane into the vapor transportchannels is provided within the heat sink. In one embodiment, membrane520 is modified to include, in addition to at least one vapor-permeableregion 523, at least one vapor-impermeable region 524. In oneembodiment, the at least one vapor-impermeable region 524 comprises aplurality of parallel-extending digits 527 that are interdigitated witha plurality of vapor-permeable areas of the at least one vapor-permeableregion 523, as illustrated in FIG. 5C. The vapor-impermeable digitsextend substantially transverse to the coolant-carrying channels 522.

In the embodiment depicted, at least one orifice 560 is provided in eachof the vapor-impermeable digits where extending over a respectivecoolant-carrying channel. Coolant is introduced into thecoolant-carrying channels through orifices 560 via liquid coolantdelivery channels 532 in fluid communication with a liquid coolant inlet540 of heat sink 503. Coolant exhaust is discharged via coolant exhaustchannels 528 extending through an opening in the membrane 520 into heatsink cap 530. Coolant exhaust channels 528 are in fluid communicationwith a coolant exhaust outlet port 542 of heat sink 503. In thisembodiment, the vapor transfer channels 531 vent vapor from the heatsink through vapor outlet port 541, and thus, both a two-phaseliquid-vapor mixture and vapor are exhausted (in one embodiment) fromthe heat sink(s). Note that in this embodiment, the orifices 560 in thevapor-impermeable digits of the membrane are jet orifices, which providejet impingement of coolant into the respective coolant-carrying channelsof the heat sink and onto a surface of the respective coolant-carryingchannel. Note also that, in this embodiment, a liquid coolant inletport, a coolant exhaust outlet port, and a vapor exhaust port areprovided in the heat sink.

As illustrated in FIGS. 5B & 5D, heat sink base 521 and heat sink cap530 are configured to accommodate an O-ring 550 to seal coolant withinthe heat sink. Liquid coolant and vapor are additionally sealed withinthe heat sink by vapor and liquid-impermeable region 524, which isprovided to extend around the perimeter of the membrane, that is, whereheld by the heat sink base and heat sink cap as illustrated in FIG. 5A.

In one embodiment, the heat sink base and heat sink cap are fabricatedof a metal material, such as copper, the coolant comprises water, andthe membrane is a porous hydrophobic membrane, such as a vapor-permeablePTFE or polypropylene material, such as the membranes available, forexample, from Sterlitech Corp., of Kent, Wash., USA, or SumitomoElectric Interconnect Products, Inc., of San Marcos, Calif., USA.

FIG. 6 depicts one embodiment of a cooled electronics apparatuscomprising a rack-level cooling apparatus with multiple heat sinkstructures, such as depicted in FIGS. 5A-5D. In this embodiment, twoheat sink structures 503 are illustrated within an electronic subsystemor node 610 of an electronics rack 600. The cooling apparatus includes amodular cooling unit 620, such as described above. Modular cooling unit620 includes a liquid-to-liquid heat exchanger 621 and a reservoir withan associated pump 622 for providing cooled liquid coolant via a coolantsupply manifold 623 and node-level supply lines 624 to the coolant inletports of the respective heat sinks 503. In the embodiment of FIGS.5A-5D, the vented coolant vapor is output from the heat sink separatefrom the coolant exhaust and is combined outside the heat sink in asingle coolant exhaust line 626 extending to a node-level condensationmodule 630, which is configured and controlled to condense coolant vaporin the exhausting coolant received via coolant exhaust line 626. Theoutput of node-level condensation module 630 is provided, via anode-level return line 631, to a rack-level coolant return manifold 625,which returns the warm liquid coolant to the modular cooling unit 620,to repeat the process. As illustrated in FIG. 6, heat exchanger 621comprises (in one embodiment) a liquid-to-liquid heat exchanger, with afacility coolant loop 601 providing facility coolant to theliquid-to-liquid heat exchanger 621. In the embodiment of FIG. 6, eachelectronic subsystem (or node) comprises a respective node-levelcondensation module 630, which (in the embodiment depicted), isliquid-cooled via a portion of the cooled coolant (or working fluid)received via the node-level coolant supply line 624. In this embodiment,a three-way control valve 640, controlled by a controller 650, divides(in one embodiment) the cooled coolant flow received via node-levelcoolant supply line 624 between the node-level condensation module 630and the heat sinks 503. One control process implemented by controller650 for controlling control valve 640 is depicted in FIG. 7.

FIG. 7 depicts one embodiment of a control process for controlling thecontrol valve(s) in a dynamically adjustable control valve (e.g.,electronic valve) implementation such as depicted in FIG. 6. Processingstarts 700 by setting (in one embodiment) the valve(s) to a 50% split ofthe cooled coolant flow between the heat sink structures and thenode-level condensation module 710. Processing waits a predefined time,such as M seconds 720, before determining whether the temperature of thecap (T_(cap)) of one or more associated electronic component(s)monitored by the controller is greater than a specified high temperaturethreshold (T_(spec,high)) 730. Note that this embodiment assumes thatone or more temperature sensors are associated with the respectiveelectronic component(s) of the associated cooled electronic structure,and that the sensed temperatures are fed back to the respectivecontroller 650, which (in one embodiment) may be a node-level controlleror a rack-level controller (see FIG. 6). If “yes”, then the valve isadjusted to increase (for example, by a set percentage (X %)) thecoolant flow to the coolant-carrying channels in the heat sinkstructure(s) 740 to provide greater cooling to the electronic component.Processing then waits the predefined time interval (e.g., M seconds) 720before again evaluating the cap temperature (T_(cap)). If the captemperature (T_(cap)) is less than the specified high temperaturethreshold (T_(spec,high)), then processing determines whether the captemperature (T_(cap)) is less than or equal to a specified lowtemperature threshold (T_(spec,low)) 750. If “yes”, then the controlvalve is automatically adjusted (for example, by the predefinedpercentage (X %)) to reduce the flow of coolant to the coolant-carryingchannels in the heat sink structure(s) 760. Thereafter, processing waitsthe defined time interval 720 before repeating the process.

Note that having the ability to adjust the ratio as needed allows thevapor quality in the cooling channels of the heat sink structures to becontrolled and provides a more uniform heat transfer coefficient andpressure drop as the thermal loads vary. This assists in reducingtwo-phase instabilities from forming in the rack as different nodesexperience different thermal loads. Initially, the cooled coolant may besplit equally between the heat sink structures and the condensationmodule, but if the temperature of one of the electronic components isbelow a lower specified limit, then less flow is diverted to themodules, and more to the condenser. The advantage of this change is areduction in the pressure drop in the node due to a lowering of theflowrate through the small-diameter cooling microchannels. If theelectronic component temperature is found to be too high, for example,above an upper specified temperature limit, indicating lack ofsufficient flow or too high of a vapor quality (which reduces the heattransfer coefficient), then the valve is adjusted to divert more coolantflow to the heat sink structures, and less to the condensation module.This assists in reducing the amount of vapor generated in themicrochannels, and reduces the electronic component temperatures.

As noted, in one embodiment, node-level condensation module 630implements direct-contact condensing using a diverted fraction of thesub-cooled coolant received via node-level supply line 624 at therespective liquid-cooled node of the electronics rack 600. Oneembodiment of such a direct-contact, node-level condensation module isdepicted in FIG. 8. In this embodiment, node-level condensation module630 comprises a multi-layer structure 800, which may be fabricated of athermally conductive material, such as copper or aluminum. The layersare configured to provide a node-level condensation module wherein thereis direct mixing between the two-phase, liquid-gaseous mixture receivedvia coolant exhaust line 626 (see FIG. 6) at a first inlet 810 ofnode-level condensation module 630 and the sub-cooled coolant receivedvia control valve 640 (FIG. 6) and node-level coolant supply line 624 ata second inlet 820 of the condensation module. As used herein,sub-cooled refers to some temperature below the saturation temperatureof the liquid at a given pressure. In this embodiment, the condensationmodule comprises one or more parallel (or serpentine) condensingchannels 811, through which the coolant exhaust passes. In theembodiment illustrated, node-level condensation module 630 furtherincludes one or more parallel (or serpentine) channels or an openchamber 821, with orifices 822, through which the cooled liquid coolantflows into the one or more condensing channels 811 to impinge and mixwith the coolant exhaust received via first inlet 810. As illustrated,the percentage of coolant vapor within the coolant exhaust drops as thecoolant exhaust flows through the one or more condensing channels 811and is mixed with the cooled coolant received into the channels viaorifices 822. The resultant warm, single-phase liquid coolant exhaust isoutput via an outlet 830 and node-level coolant return line 631 to therack-level return manifold 625, for return to the modular cooling unit620, to repeat the process.

Referring collectively to FIGS. 5A-8, operationally, at low heat fluxes,coolant impinges on the coolant-carrying channel surfaces of the heatsink base and flows down the coolant-carrying channels thereof as asingle-phase liquid to the coolant exhaust plenum at either end of thechannels. The liquid-impermeable nature of the vapor-permeable membranestops the liquid from leaking from the coolant-carrying channels throughthe pores of the membrane into the vapor transport channels in the heatsink cap. The liquid impingement has a higher heat transfer coefficient,and the relatively shorter flow lengths facilitate reducing flowpressure drop, and may maintain better temperature uniformity comparedwith coolant delivered parallel to the heated surface. The liquid flowsto the external cooling apparatus (as shown in FIG. 6), for processingthrough the node-level condensation module 630 and return via therack-level coolant return manifold 625 to the modular cooling unit.Within the modular cooling unit, the heated coolant is cooled by theheat exchanger, with heat being rejected to the facility coolant passingthrough the heat exchanger. The cooled liquid coolant is then pumpedback to the nodes of the electronics rack, and in particular, to flowthrough the heat sinks and node-level condensation modules, in a mannersuch as described above.

At higher heat fluxes, a portion of the impinging coolant vaporizeswithin the coolant-carrying channels, with a liquid and vapor mixtureflowing down the length of the channels. However, the vapor phase mayalso egress through the vapor-permeable region(s) of the membrane intothe vapor transport channels of the heat sink cap, leaving a relativelyliquid-rich coolant exhaust flowing in the coolant channels. This localremoval of the vapor helps maintain a high heat transfer coefficient,reduces the pressure drop, and reduces dryout within the heat sink. Theseparated vapor can then be reintroduced into the coolant exhaust (e.g.,a two-phase exhaust mixture) exiting from the heat sink structure. Thetwo-phase coolant effluent flows to the node-level condensation module,where the vapor is condensed, for example, with the assistance ofsub-cooled coolant provided to the condenser. In the node-levelcondensation module, the diverted, sub-cooled coolant is, for example,mixed or sprayed directly into the two-phase mixture from the heat sinkstructure(s), leading to direct-contact condensation of the vapor. Warm,single-phase fluid then leaves the condensation module for return to themodular cooling unit, to be chilled and pumped back to the nodes torepeat the process.

Advantageously, the node-level condensation module described herein:reduces pressure drop in channels, tubings and manifolds by providing acondensed, single-phase liquid flow from the node to the rack-levelstructures (instead of a two-phase mixture); eliminates the need forrack-level vapor separation and condensing; and in the case of asingle-fluid, direct-contact condensing approach such as described abovein connection with FIG. 8, there is the possibility of incorporating thecooling solution into existing single-phase platforms, since therack-level architecture is similar; and in the case of single-fluid,direct-contact condensing, the potential exists for fine-tuning coolingperformance through the use of electronic valving 640 (FIG. 6) such thatthe liquid flowrate to the heat sink channels and/or the condensermodule is adjustable (for example, as described above in connection withthe control process of FIG. 7).

By way of further example, FIGS. 9-11 depict an alternate cooledelectronics apparatus comprising multiple heat sink structures, such asdepicted in FIGS. 5A-5D. In this embodiment, two heat sink structures503 are illustrated within an electronic subsystem or node 910 of anelectronics rack 900. The cooling apparatus includes a modular coolingunit 620, such as described above. Modular cooling unit 620 includes aliquid-to-liquid heat exchanger 621 and a reservoir with an associatedpump 622 for providing cooled liquid coolant via a coolant supplymanifold 623 and node-level supply lines 624 to the coolant inlet portsof the respective heat sinks 503. The vented coolant vapor is outputfrom the heat sink structures 503, in one embodiment, separate from thecoolant exhaust, and is combined outside the heat sink structures withina single coolant exhaust line 626 as a two-phase mixture. Coolantexhaust line 626 is coupled to a node-level condensation module 930,which in this example, is an indirect-cooled, condensation module. Forexample, the condensation module embodiment of FIG. 9 employs atwo-fluid condensation scheme, wherein (by way of example) aliquid-cooled structure within the condensation module may be employedto condense coolant vapor received into the module via coolant exhaustline 626.

In the two-fluid condensation approach of the cooled electronicsapparatus of FIG. 9, a cold (or chilled) facility coolant is receivedvia facility supply and return manifolds 931, 932 and node-levelfacility coolant supply and return lines 933, 934 to the individualnode-level condensation modules 930 associated with the one or moreelectronic subsystems or nodes 910 of the electronics rack 900. Thecooled electronics apparatus further includes a control valve 940controlled by a controller 950. In this embodiment, control valve 940divides the facility coolant flow received via a rack-level facilitycoolant loop, comprising a facility coolant supply line 936 and afacility coolant return line 937, between the liquid-to-liquid heatexchanger 621 and the facility coolant supply manifold 931. One controlprocess implanted by controller 950 for controlling control valve 940 isdepicted in FIG. 10.

FIG. 10 depicts one embodiment of a control process for controlling thecontrol valve(s) in a dynamically adjustable control valveimplementation such as depicted in FIG. 9. Processing starts 1000 bysetting (in one embodiment) the valve(s) to a 50% split of the cooledfacility coolant flow between the modular cooling unit and thenode-level condensation modules 1010. Processing waits a predefinedtime, such as M seconds 1020, before determining the node-level exitvapor quality, based on the current thermal load of the node, systemcoolant flowrate and change in temperature (ΔT_(SC)), and facilitycoolant flowrate and change in temperature (ΔT_(FC)) 1030. Appropriatelypositioned flowrate and temperature sensors facilitate thesedeterminations. Processing then determines whether the determined vaporquality is at or above a high-quality threshold 1040, and if “yes”, thenthe valve is adjusted to increase (for example, by a set percentage (X%)) the facility coolant flow to the node-level condensation modules1050 to provide greater cooling to the condenser, and thus, greatervapor-condensing. Processing then waits the predetermined time interval(e.g., M seconds) before again evaluating the system coolant flowrateand system coolant change in temperature (ΔT_(SC)), as well as facilitycoolant flowrate and facility coolant change in temperature (ΔT_(FC))1030. If the vapor quality is below the upper threshold, then processingdetermines whether the vapor quality is at or below a lower qualitythreshold 1060. If “yes”, then the control valve is automaticallyadjusted (for example, by the predefined percentage (X %)) to reduce theflow of facility coolant to the node-level condensation modules 1070.Thereafter, processing waits the defined time interval 1020 beforerepeating the process. The vapor quality used in the rack level controlimplementation is, in one embodiment, the maximum determined vaporquality among the nodes.

FIG. 11 depicts one embodiment of a node-level condensation module 930embodying an indirect-cooling approach to condensing coolant vaporwithin the module. In the illustrated embodiment, node-levelcondensation module 930 comprises multiple thermally conductive layers1100, fabricated, for example, of copper or aluminum. A first coolantinlet 1110 is coupled to receive the two-phase coolant exhaust mixturefrom the one or more heat sink structures. This two-phase coolantexhaust mixture is passed through one or more condensing channels, whichmay comprise parallel or serpentine channels within the condensationmodule. Warm liquid coolant exhaust is output via coolant outlet 1112for return via node-level return line 631 to rack-level coolant returnmanifold 625, to repeat the cooling process. The vapor-condensation isindirect, in that heat is removed to the facility coolant via conductionto a liquid-cooled structure comprising one or more cooling channels1121 overlying the one or more condensing channels 1111. As illustrated,a second coolant inlet 1120 is coupled to receive cooled facilitycoolant, for example, via node-level facility coolant supply line 933from facility coolant supply manifold 931. The facility coolant isexhausted from the condensation module via a second, facility coolantoutlet 1122 and returned via node-level facility return line 934 to therack-level facility coolant manifold 932.

Referring collectively to FIGS. 9-11, note that all sub-cooled systemcoolant in this implementation is pumped directly to the heat sinkstructures for use in cooling the associated electronics component(s) ofthe electronic subsystem. The vented vapor and two-phase coolant exhaustare merged within the electronic subsystem and plumbed to the node-levelcondensation module 930, where the two-phase mixture is condensed into asingle-phase liquid through heat exchange with the facility coolantpassing through the liquid-cooled structure of the condensation module.This facility coolant, for example, may be chilled house water obtainedby splitting off a fraction of the chilled facility coolant pumped tothe modular cooling unit for sub-cooling of the system or workingcoolant. This arrangement results in additional plumbing connections ateach node, and the need for additional manifolding. However, thearrangement has the advantage of requiring the smallest coolant flow,and potentially pressure drop, since all of the coolant is used indirect cooling, instead of being split at the node to cool both the heatsink structures and the condensation module.

In operation, at low heat loads, the fluid exiting the heat sinkstructures may be warm, single-phase liquid, in which case condensationwithin the condensation module is unnecessary, and all of the facilitycoolant can be diverted to the modular cooling unit to maximizesub-cooling. However, as the heat load rises, vapor quality increaseswithin the nodes, and it is desirable to condense the coolant vapor atthe node-level to ensure that only warm liquid departs from the nodeback to the modular cooling unit. In this case, the electronic valve 940diverts more facility coolant flow to the condensation modules withinthe nodes, and less to the modular cooling unit. In the implementationof FIG. 9, only a single, primary diverting control valve is depicted,with the fraction being set by the maximum thermal load experiencedwithin the electronics rack.

FIG. 12 depicts an alternate implementation of a cooled electronicsapparatus comprising multiple heat sink structures, such as depicted inFIGS. 5A-5D, and a node-level condensation module such as depicted inFIGS. 9 & 11, and described above. In this implementation, the singlecontrol valve 940 of FIG. 9 is replaced by a plurality of node-levelcontrol valves 1210 coupled in fluid communication with the node-levelfacility coolant supply lines 933 coupling the respective node-levelcondensation module 930 and the facility coolant supply manifold 931. Inone embodiment, the node-level control valves 1210 set the facilitycoolant flowrate through the respective node-level condensation module.In this embodiment, a fixed percentage of the facility coolant is splitbetween the liquid-to-liquid heat exchanger 621 of the modular coolingunit 620 and the facility coolant supply manifold 931 of the electronicsrack 1200. A single controller or multiple controllers within theelectronics rack could be coupled to automatically control the pluralityof control valves 1210, for example, to independently adjust facilitycoolant flow through the respective node-level condensation modules 930,as required. In this implementation, only node-level condensationmodules that require cooling to condense coolant vapor would receive thefacility-chilled coolant. For example, the electronic valve(s) may beset based on measurements of the respective heat load (or power) withina given electronic subsystem, from which coolant vapor quality may beestimated for a given system coolant and facility coolant flowrate, andtemperature changes (ΔT_(SC) & ΔT_(FC)). The control process could besimilar to that depicted by FIG. 10 and implemented at the node level byusing vapor quality determined at each individual node.

The above-described, cooled electronics apparatuses may employ differentheat sink structures than that depicted above in FIGS. 5A-5D By way offurther example, FIGS. 13 & 14 depict two alternative cooled electronicstructures which may be coupled at the node-level to a condensationmodule, such as described herein.

As noted, FIG. 13 depicts an alternate embodiment of a cooledelectronics structure, generally denoted 1300, in accordance with one ormore aspects of the present invention. Cooled electronic structure 1300is similar to cooled electronic structure 500 of FIGS. 5A-5D, except thelayers that make up heat sink 1303 of FIG. 13 are modified from thelayers that make up heat sink 503 (see FIG. 5A).

Specifically, as shown in FIG. 13, cooled electronic structure 1300includes, in this embodiment, electronic component 510, such as anelectronic module, mounted to printed circuit board 501, with anassociated back plate 502. Heat sink 1303 is mechanically coupled viasecuring mechanisms 505 to back plate 502 of printed circuit board 501,which provide compressive loading of heat sink 1303 to electroniccomponent 510. Electronic component 510 includes, in this embodiment,integrated circuit chip 511 connected to chip carrier or substrate 513via a first plurality of solder ball connections 512. Substrate 513 iselectrically connected to printed circuit board 501 via a secondplurality of solder ball connections 514. A thermally conductive cap 516is interfaced to integrated circuit chip 511 via first thermal interfacematerial 515, and to heat sink 1303 via second interface material 517,which may be the same or different interface materials.

Heat sink 1303 is again a multilayer heat sink with a heat sink base521, a membrane structure 520, and a heat sink cap 530. Heat sink base521 comprises one or more coolant-carrying channels 522, each of whichmay comprise a microchannel structure, such as described above. Inoperation, heat from the electronic component is rejected to coolantwithin the coolant-carrying channels in the heat sink base 521, causingboiling of the coolant.

As illustrated in FIG. 13, various regions of the coolant-carryingchannels are capped by at least one vapor-permeable region 523 ofmembrane 520. Disposed over these regions are vapor transport channels531 formed in heat sink cap 530. Thus, localized venting of vapordirectly from the coolant-carrying channels, across the vapor-permeablemembrane into the vapor transport channels is provided within the heatsink. In one embodiment, membrane 520 is modified to include, inaddition to the at least one vapor-permeable region 523, at least onevapor-impermeable region 524. In one embodiment, the at least onevapor-impermeable region 524 comprises a plurality of parallel-extendingdigits that are interdigitated with a plurality of vapor-permeable areasof the at least one vapor-permeable region 523. The vapor-impermeabledigits extend substantially transverse to the coolant-carrying channels522.

In the embodiment depicted, at least one orifice 560 is provided in eachof the vapor-impermeable digits where extending over a respectivecoolant-carrying channel. Coolant is introduced into thecoolant-carrying channels through orifices 560 via liquid coolantdelivery channels 532, which in one embodiment, are interdigitated withthe vapor transport channels 531 within the heat sink cap 530. Liquidcoolant delivery channels 532 are in fluid communication with a liquidcoolant inlet port 540 of heat sink 1303. Coolant exhaust is dischargedvia coolant exhaust channel(s) 529 through a coolant exhaust outlet port1340. In this embodiment, the vapor transport channels 532 couple to thecoolant exhaust channel(s) 529 within the heat sink and a two-phasecoolant exhaust is output via coolant exhaust outlet port 1340.

As with the cooled electronic structure embodiment of FIGS. 5A-5D, heatsink base 521 and heat sink cap 530 are configured to accommodate, inthis embodiment, an O-ring 550 to seal coolant and vapor within the heatsink. Coolant and vapor are additionally sealed within the heat sink bythe vapor-impermeable region 524 defined around the perimeter of themembrane 520, that is, where held by the heat sink base and heat sinkcap, as illustrated in FIG. 13.

In one embodiment, the heat sink base and heat sink cap are fabricatedof a metal material, such as copper, the coolant comprises water, andthe membrane is a vapor-permeable, liquid-impermeable membrane (exceptfor the jet orifices), such as a vapor-permeable PTFE or polypropylenematerial.

FIG. 14 depicts another embodiment of a cooled electronic structure1400, in accordance with one or more aspects of the present invention.The cooled electronic structure of FIG. 14 is similar to cooledelectronic structure 500 of FIGS. 5A-5D, except that the single-layermembrane 523 of FIGS. 5A-5D is replaced by a multilayer structurecomprising (in one embodiment) a vapor-permeable membrane 1410positioned between two masking plates 1420, as illustrated in FIG. 14.

Specifically, as shown in FIG. 14, the cooled electronic structureincludes, in this embodiment, electronic component 510, such as anelectronic module, mounted to a printed circuit board 501, with anassociated back plate 502. Heat sink 1403 is mechanically coupled viasecuring mechanisms 505 to back plate 502 of printed circuit board 501,which provide compressive loading of heat sink 1403 to electroniccomponent 510. Electronic component 510 includes, in this embodiment,integrated circuit chip 511 connected to chip carrier or substrate 513via a first plurality of solder ball connections 512. Substrate 513 iselectrically connected to printed circuit board 501 via a secondplurality of solder ball connections 514. A thermally conductive cap 516is interfaced to integrated circuit chip 511 via first thermal interfacematerial 515, and to the heat sink 1403 via second thermal interfacematerial 517, which may be the same or different interface materials.

Heat sink 1403 is again a multilayer heat sink, with a heat sink base521, a multilayer membrane structure comprising masking plates 1420, andvapor-permeable membrane 1410, and a heat sink cap 530. Heat sink base521 comprises one or more coolant-carrying channels 522, each of whichmay comprise a microchannel structure, such as described above. Inoperation, heat from the electronic component is rejected to coolantwithin the coolant-carrying channels in the heat sink base 521, causing(in one mode) boiling of the coolant.

As illustrated in FIG. 14, various regions of the coolant-carryingchannels 522 are capped by vapor-permeable membrane 1410, which ispositioned in between masking plates 1420 and exposed to thecoolant-carrying channels via open regions in masking plates 1420. Theseexposed regions of vapor-permeable membrane 1410 align to vapor transferchannels 531, which vent vapor egressing from the coolant-carryingchannels, as explained above.

Jet orifices or nozzles are defined in the multi-layer membranestructure via aligned through-holes in masking plates 1420, andvapor-permeable membrane 1410. As explained above, these jet orificesinject coolant from liquid coolant delivery channels 532 into thecoolant-carrying channels 522 in heat sink base 521.

In one embodiment, masking plates 1420 comprise metal masking plates,which may be epoxied, soldered or press-fitted to heat sink base 521 andheat sink cap 530. Additionally, masking plates 1420 may be epoxied tothe vapor-permeable membrane 1410 for better sealing. Note also that theopen regions in the masking plate 1420 exposed to the coolant-carryingchannels 522 operate as vapor traps, where vapor collects between thechannels and the membrane. This further facilitates egress of the vaporacross the membrane into the vapor transport channels 532. Note further,that in the depicted multilayer membrane structure embodiment, thevapor-permeable membrane of FIG. 14 need not have a vapor-impermeableregion, such as in the embodiments described above. Note also that othermultilayer membrane structure embodiments may alternatively be employedwith a heat sink structure as described herein. For example, a singlemasking plate could be employed with the vapor-permeable membrane, ifdesired.

Those skilled in the art will note from the above discussion that theheat sink structures described herein include a heat sink base whichcomprises one or more coolant-carrying channels. In one embodiment,these coolant-carrying channels have sub-millimeter hydraulic diameters,and also are referred to herein as “microchannels”. Such small channelshelp increase the surface area, as well as the single-phase heattransfer coefficient of the coolant within the channels. The channelscan be made via chemical etching or mechanical methods, such as skivingor end-milling. In one embodiment, the heat sink is fabricated ofcopper, due to its high heat transfer coefficient and relatively simplemachineability. However, other materials, such as aluminum and siliconare also suitable, though may have disadvantages in terms of thermalconductivity, fragility and machineability.

The second layer of the heat sink comprises a vapor-permeable membrane,such as a porous, hydrophobic membrane, in the case where the coolantcomprises water. Examples of micro/nano-porous, natively hydrophobicmembranes include polypropylene, PTFE, and nylon. Natively hydrophilicmaterials, such as porous glass, porous silicon, porous aluminum andporous organic materials could also be used, but require a liquid-phobiccoating to prevent liquid from leaking into the vapor channels. Theporous membrane is prepared such that the regions with the nozzles ororifices, as well as the edges of the membrane, are hardened andnon-porous to provide better nozzle definition as well as edge sealing.The membrane can be patterned using a variety of techniques, such as hotpress (wherein a heated master is pressed onto the porous membrane toplastically deform it and close the pores in the desired regions),laminating with a non-porous material (one example of which is laminatedporous PTFE, where the laminate is made of non-porous polypropylene), orepoxy/photoresist infiltration (where epoxy could be used to selectivelyclose the pores and provide additional mechanical stiffness in desiredregions).

In an alternate embodiment, the second layer of the heat sink mightcomprise a multilayer membrane structure, for example, such as depictedin FIG. 14, and described above. In such a multilayer structure, themembrane may be a vapor-permeable membrane, for instance, a porous,hydrophobic membrane, in the case where the coolant comprises water.Additionally, the masking plate may be fabricated of variousvapor-impermeable materials, with metal being one example.

The third layer of the heat sink, that is, the heat sink cap, comprisesrelatively larger liquid and vapor channels which help distribute thefluid from and to the inlet and outlet ports of the heat sink. In orderto minimize the pressure drop in these channels, the hydraulic diameteris maintained relatively large. A large hydraulic diameter also reducesthe pressure drop when the vented vapor is reintroduced to the coolanteffluent (which may be a two-phase effluent) at the heat sink level. Theheat sink cap can be made of copper or aluminum or any other materialwith a similar coefficient of thermal expansion (CTE) as that of theheat sink base to avoid excessive thermal stresses developing.

The coolant (or working fluid) should be compatible with the selectedmembrane, thus requiring specific fluid/membrane combinations. Examples,of coolants (or working fluids) include: water at sub-ambient pressures,dielectric fluids at atmospheric pressure, and refrigerants at higherpressures.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprise” (andany form of comprise, such as “comprises” and “comprising”), “have” (andany form of have, such as “has” and “having”), “include” (and any formof include, such as “includes” and “including”), and “contain” (and anyform contain, such as “contains” and “containing”) are open-endedlinking verbs. As a result, a method or device that “comprises”, “has”,“includes” or “contains” one or more steps or elements possesses thoseone or more steps or elements, but is not limited to possessing onlythose one or more steps or elements. Likewise, a step of a method or anelement of a device that “comprises”, “has”, “includes” or “contains”one or more features possesses those one or more features, but is notlimited to possessing only those one or more features. Furthermore, adevice or structure that is configured in a certain way is configured inat least that way, but may also be configured in ways that are notlisted.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below, if any, areintended to include any structure, material, or act for performing thefunction in combination with other claimed elements as specificallyclaimed. The description of the present invention has been presented forpurposes of illustration and description, but is not intended to beexhaustive or limited to the invention in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the invention.

What is claimed is:
 1. A method of facilitating extraction of heat froman electronic subsystem, the method comprising: providing at least oneheat sink configured to cool at least one electronic component of theelectronic subsystem of an electronics rack, the at least one heat sinkcomprising at least one coolant-carrying channel configured for coolantto flow therethrough, the coolant providing two-phase cooling to the atleast one electronic component and being discharged from the at leastone heat sink as coolant exhaust with coolant vapor; providing anode-level condensation module in association with the electronicsubsystem, and coupling in fluid communication the at least one heatsink and the node-level condensation module, the node-level condensationmodule receiving the coolant exhaust from the at least one heat sink,the node-level condensation module being liquid-cooled and facilitatingcondensing of the coolant vapor in the coolant exhaust; andautomatically controlling at least one of the liquid-cooling of the atleast one heat sink or the liquid-cooling of the node-level condensationmodule.
 2. The method of claim 1, further comprising providing a controlvalve directing coolant flow to the at least one heat sink and to thenode-level condensation module for the liquid-cooling thereof, thecontrol valve being automatically controlled by the controller.
 3. Themethod of claim 2, wherein the control valve divides liquid coolant flowreceived at the electronic subsystem from a rack-level coolant supplymanifold of the electronics rack between the at least one heat sink andthe node-level condensation module, the control valve beingautomatically controlled by the controller to automatically adjust afraction of received coolant flow provided to the at least one heat sinkbased on a temperature associated with the at least one electroniccomponent.
 4. The method of claim 1, wherein the node-level condensationmodule comprises at least one condensing channel, and provides directcooling of the coolant vapor within the at least one condensing channelby injecting cooled coolant in direct contact with the coolant vaporwithin the at least one condensing channel.
 5. The method of claim 1,wherein the node-level condensation module is coupled in fluidcommunication with a rack-level coolant return manifold of theelectronics rack, and provides warm, single-phase liquid coolant to therack-level coolant return manifold, and wherein the node-levelcondensation module is spaced from the at least one heat sink within theelectronic subsystem.
 6. The method of claim 1, wherein one heat sink ofthe at least one heat sink comprises: a thermally conductive structurecomprising at least one coolant-carrying channel; and a membranestructure associated with the at least one coolant-carrying channel, themembrane structure comprising at least one vapor-permeable region, atleast a portion of the at least one vapor-permeable region overlying aportion of the at least one coolant-carrying channel and facilitatingremoval of vapor from the at least one coolant-carrying channel, and themembrane structure further comprising at least one orifice coupled toinject coolant onto at least one surface of the at least onecoolant-carrying channel intermediate ends of the at least onecoolant-carrying channel.
 7. The cooling apparatus of claim 1, whereinthe coolant is a first coolant, the coolant vapor is a first coolantvapor, and the node-level condensation module is liquid-cooled via asecond coolant, the node-level condensation module comprising aliquid-to-liquid heat exchanger providing conductive heat transfer fromthe first coolant to the second coolant to facilitate condensing of thefirst coolant vapor in the node-level condensation module.
 8. The methodof claim 7, further comprising providing a coolant distribution unitassociated with the electronics rack, the coolant distribution unitproviding cooling to the first coolant by transferring heat from thefirst coolant to the second coolant, and wherein the second coolantflows in parallel to the node-level condensation module and the coolantdistribution unit.
 9. The method of claim 8, further comprisingproviding a node-level control valve controlling a flowrate of thesecond coolant to the node-level condensation module, the node-levelcontrol valve being automatically controlled by the controller based ona vapor quality of the first coolant.
 10. The method of claim 8, furthercomprising providing a rack-level control valve for directing fractionsof the second coolant to the node-level condensation module and thecoolant distribution unit, the rack-level control valve beingautomatically controlled by the controller to adjust the fraction of thesecond coolant flowing to the node-level condensation module based on avapor quality of the first coolant.